Multi-market calibration of convenience panel data to reduce behavioral biases

ABSTRACT

Example methods, apparatus, systems and articles of manufacture to implement calibration of convenience panel data to reduce behavioral bias are disclosed. Disclosed example apparatus include a distribution estimator to determine a first behavioral distribution for first convenience panel data associated with a first market and a measurement period, determine a second behavioral distribution for second convenience panel data associated with a second market and the measurement period, and determine a third behavioral distribution for probabilistic panel data associated with the second market and the measurement period. Disclosed example apparatus also include a distribution calibrator to calibrate the first behavioral distribution determined for the first convenience panel data associated with the first market based on (i) the second behavioral distribution determined for the second convenience panel data associated with the second market and (ii) the third behavioral distribution determined for the probabilistic panel data associated with the second market.

RELATED APPLICATION(S)

This patent arise from a continuation of U.S. patent application Ser.No. 16/721,474, which was filed on Dec. 19, 2019, and is titled“MULTI-MARKET CALIBRATION OF CONVENIENCE PANEL DATA TO REDUCE BEHAVIORALBIASES,” which is a continuation of International Patent Application No.PCT/US19/58829, which was filed on Oct. 30, 2019, and is titled“MULTI-MARKET CALIBRATION OF CONVENIENCE PANEL DATA TO REDUCE BEHAVIORALBIASES,” which claims the benefit of U.S. Provisional Application No.62/753,657, which was filed on Oct. 31, 2018, and is titled“MULTI-MARKET PROBABILISTIC SUBSTITUTION USING ITERATIVE LEARNING TOESTIMATE BEHAVIORAL BIASES.” Priority to U.S. patent application Ser.No. 16/721,474, International Patent Application No. PCT/US19/58829 andU.S. Provisional Application No. 62/753,657 is claimed. U.S. patentapplication Ser. No. 16/721,474, International Patent Application No.PCT/US19/58829 and U.S. Provisional Application No. 62/753,657 areincorporated herein by reference in their respective entireties.

FIELD OF THE DISCLOSURE

This disclosure relates generally to audience measurement and, moreparticularly, to multi-market calibration of convenience panel data toreduce behavioral biases.

BACKGROUND

Some audience measurement systems for measuring audiences of onlinemedia (e.g., website visits, online advertisements, online programming,etc.) utilize impression data obtained from probabilistic panels and/orconvenience panels. Probabilistic panels can include individuals,households, etc., who are recruited (e.g., via telephone and/orin-person interviews) to meet specified demographic targets (e.g.,corresponding to a demographic distribution of a target population).Probabilistic panels can provide an accurate, granular truth data setthat represents behaviors of individuals (and thus demographic groups)in the panel. However, probabilistic panels may have small samples sizesand may be expensive to maintain. In contrast, convenience panels caninclude individuals, households, etc., who are included in a panelopportunistically, such as in response to an online prompt to join thepanel. Behavioral information for panelists in a convenience panel maybe known or unknown, can be biased towards some behaviors, and/or notrepresentative of a target population. However, convenience panels mayhave large sample sizes and may be relatively inexpensive to maintain.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example audience measurement systemimplementing multi-market calibration of convenience panel data usingprobabilistic substitution with iterative learning to reduce behavioralbias in accordance with teachings of this disclosure.

FIG. 2 is a block diagram of a second example audience measurementsystem implementing multi-market calibration of convenience panel datato reduce behavioral bias in accordance with teachings of thisdisclosure.

FIG. 3 is a block diagram of an example behavioral distributionestimator included in the example audience measurement system of FIG. 2.

FIG. 4 is a block diagram of an example behavioral distributioncalibrator included in the example audience measurement system of FIG. 2.

FIGS. 5A-5D illustrate a first example technique for calibratingbehavioral distributions determined for convenience panel dataassociated with a first market based on behavioral distributionsdetermined for probabilistic panel data associated with the firstmarket.

FIGS. 6A-6D illustrate a second example technique for determiningbehavioral distributions for convenience panel data associated with thefirst market.

FIGS. 7A-7D illustrate a third example technique for iterativelycalibrating behavioral distributions determined for convenience paneldata associated with a first market based on an initial behavioraldistribution determined for probabilistic panel data associated with asecond market different from the first market.

FIGS. 8A-8D illustrate a fourth example technique for calibratingbehavioral distributions determined for convenience panel dataassociated with a first market based on behavioral distributionsdetermined for probabilistic panel data and convenience panel dataassociated with a second market different from the first market.

FIGS. 9A-9C depict a comparison of the first example techniqueillustrated in FIGS. 5A-5D, the second example technique illustrated inFIGS. 6A-6D, and the fourth example technique illustrated in FIGS.8A-8D.

FIG. 10 is a flowchart representative of example machine readableinstructions that may be executed to implement the example audiencemeasurement system of FIG. 1 .

FIG. 11 is a flowchart representative of example machine readableinstructions that may be executed to implement the example audiencemeasurement system of FIG. 2 .

FIG. 12 is a block diagram of an example processor platform structuredto execute the example machine readable instructions of FIGS. 10 and/or11 to implement the example audience measurement systems of FIGS. 1and/or 2 .

The figures are not to scale. In general, the same reference numberswill be used throughout the drawing(s) and accompanying writtendescription to refer to the same or like parts, elements, etc.

Descriptors “first,” “second,” “third,” etc., are used herein whenidentifying multiple elements or components which may be referred toseparately. Unless otherwise specified or understood based on theircontext of use, such descriptors are not intended to impute any meaningof priority or ordering in time but merely as labels for referring tomultiple elements or components separately for ease of understanding thedisclosed examples. In some examples, the descriptor “first” may be usedto refer to an element in the detailed description, while the sameelement may be referred to in a claim with a different descriptor suchas “second” or “third.” In such instances, it should be understood thatsuch descriptors are used merely for ease of referencing multipleelements or components.

DETAILED DESCRIPTION

Example methods, apparatus, systems and articles of manufacture (e.g.,physical storage media) to implement multi-market calibration ofconvenience panel data to reduce behavioral bias are disclosed herein.As noted above, some audience measurement systems for measuringaudiences of online media (e.g., website visits, online advertisements,online programming, etc.) utilize impression data obtained fromprobabilistic panels and/or convenience panels. Probabilistic panels caninclude individuals, households, etc., who are recruited (e.g., viatelephone and/or in-person interviews) to meet specified demographictargets (e.g., corresponding to a demographic distribution of a targetpopulation). Panelists in a probabilistic panel may agree to providedetailed demographic information to an audience measurement entity (AME)managing the panel. Panelists in a probabilistic panel may also agree tothe use of metering devices and/or installed metering applications tomonitor their access of and/or interaction with online media. Thus,probabilistic panels can provide an accurate, granular truth data setthat represents behaviors of individuals in the panel. However,probabilistic panels may have small samples sizes and may be expensiveto maintain. In contrast, convenience panels can include individuals,households, etc., who are included in a panel opportunistically, such asin response to an online prompt, banner, etc., to join the panel. Forexample, websites serving the online media may prompt users to agree tohave their access information provided to a convenience panel, and mayprompt the users to enter demographic information. In some examples,demographic information for panelists in a convenience panel may beknown, unknown and/or unverified, may be biased towards somedemographics, and/or may not be representative of a target population.However, convenience panels may have large sample sizes and may berelatively inexpensive to maintain.

To take advantage of the relative strengths of probabilistic panel dataand convenience panel data while deemphasizing their respectiveweaknesses, some audience measurement systems combine probabilisticpanel data and convenience panel data to obtain an overall measurementdata set or, in other words, a target panel data set for a given market.For example, some such audience measurement systems combine theprobabilistic panel data and convenience panel data based ondemographics and/or geographic information, such as through demographicweighting or geodemographic weighting. To perform such combining, somesuch audience measurement systems determine demographic distributionsfor a population in a given market based on both the availableprobabilistic panel data and the available convenience panel data forthe given market. Because the probabilistic panel data is intended to bean accurate representation of the population in the given market, it canbe combined with the convenience panel data to reduce or otherwisecalibrate for the potential bias associated with the convenience paneldata while taking advantage of the increased sample size afforded by theconvenience data panel. However, as noted above, probabilistic panelscan be expensive to maintain. Also, in some markets, it may be difficultto recruit an adequate number of panelists to be representative of thetarget population and, thus, some demographic categories may beunderrepresented such that demographic weighting alone (or incombination with location-based weighting) is unable to adequatelyreduce the bias associated with the convenience panel data. Thus, insome markets, the probabilistic panel data may not have adequate quality(e.g., in terms of representing the demographics of the targetpopulation with respective target sample sizes) to correct the biasassociated with the convenience panel data in those markets.

Example audience measurement systems disclosed herein provide technicalsolutions to the technical problems associated with unrepresentativeprobabilistic panel data for a given market. In particular, disclosedexample audience measurement systems implement multi-market calibrationof convenience panel data to reduce behavioral bias. To solve theproblem of unrepresentative probabilistic panel data for a given market,some disclosed example audience measurement systems utilize behaviorcharacteristics, such as in the form of behavior distributions, tocalibrate convenience panel data. For example, some disclosed exampleaudience measurement systems utilize a behavioral distributiondetermined for probabilistic panel data from another market (e.g.,referred to herein as a calibration market) as an initial seed tocalibrate a behavioral distribution determined for convenience paneldata associated with the given target market. Such disclosed exampleaudience measurement systems then iteratively combine the calibratedbehavioral distribution determined from a prior iteration (which wasinitially seeded with the probabilistic panel behavior distributiondetermined for the calibration market) with a new behavior distributiondetermined for the convenience panel data from the given target marketto determine a new calibrated behavioral distribution for theconvenience panel data in the given target market. The resultingcalibrated behavioral distribution can then be output for use as inputto downstream audience measurement processing stages that utilizeaudience distributions determined for the given target market. Thecalibrated behavioral distribution determined for the current processingiteration is also used as the initial behavioral distribution for thenext processing iteration. In this way, a higher quality probabilisticpanel behavioral distribution for a different market (e.g., thecalibration market) can be used to seed the calibration process used tocalibrate the convenience panel data in the given target market, withany potential differences between the characteristics of theprobabilistic panel for the other calibration market and thecharacteristics of the given target market diminishing over time witheach processing iteration.

Some disclosed example audience measurement systems calibrate theconvenience panel data for a given market and measurement interval(e.g., monthly or some other time period) using probabilistic panel dataand convenience panel data obtained for that measurement interval fromone or more other calibration markets. For example, and as disclosed infurther detail below, some such audience measurement systems maydetermine, for a given measurement interval, a behavioral distributionfor the convenience panel data from the given target market, abehavioral distribution for the probabilistic panel data from thecalibration market, and a behavioral distribution for the conveniencepanel data from the calibration market. As described in further detailbelow, a behavioral distribution may represent counts of individuals(such as counts of panelists in a probabilistic panel, counts ofpanelists in a convenience panel, counts of people in a population ofinterest, etc.) associated with different behavior categories. Somedisclosed example audience measurement systems then calibrate, for agiven measurement interval, the behavioral distribution for theconvenience panel data from the given target market using the behavioraldistribution for the probabilistic panel data from the calibrationmarket and the behavioral distribution for the convenience panel datafrom the calibration market. Some such audience measurement systems maythen output the resulting calibrated behavioral distribution for theconvenience panel data from the given target market, and/or apply thecalibrated behavioral distribution to the convenience panel data togenerate audience measurement data associated with online mediaaccess/exposure in the target market.

Turning to the figures, a block diagram of an example audiencemeasurement system 100 implementing multi-market calibration ofconvenience panel data to reduce behavioral bias in accordance withteachings of this disclosure is illustrated in FIG. 1 . The exampleaudience measurement system 100 includes an example probabilistic panelinterface 105, an example convenience panel interface 110, an examplebehavioral distribution estimator 115, an example calibrationdistribution selector 120, an example behavioral distribution calibrator125 and an example target convenience panel data calibrator 130. In theillustrated example, the probabilistic panel interface 105 is structuredto obtain probabilistic panel data by interfacing (e.g., via a network)with a first impression monitoring system generating online mediaimpression data for panelists in a probabilistic panel of a targetmarket and/or generating online media impression data for panelists inprobabilistic panel(s) of one or more calibration markets. Theimpression data may correspond to, for example, counts of one or moredifferent websites or web pages visited by respective ones of thepanelists in a given probabilistic panel, counts of one or moredifferent online advertisements viewed by respective ones of thepanelists in a given probabilistic panel, counts of one or moredifferent online media programs accessed by respective ones of thepanelists in a given probabilistic panel, etc.

In the illustrated example, the convenience panel interface 110 isstructured to obtain convenience panel data by interfacing (e.g., via anetwork) with a second impression monitoring system generating onlinemedia impression data for panelists in a convenience panel of a targetmarket and/or generating online media impression data for panelists inconvenience panel(s) of one or more calibration markets. The impressiondata may correspond to, for example, counts of one or more differentwebsites visited by respective ones of the panelists in a givenconvenience panel, counts of one or more different online advertisementsviewed by respective ones of the panelists in a given convenience panel,counts of one or more different online media programs accessed byrespective ones of the convenience in a given convenience panel, etc. Insome examples, the convenience panel data obtained by the conveniencepanel interface 110 is limited to impression data that is aggregatedacross the panelists in the convenience panel. In some examples, themarket for which convenience panel data is obtained by the conveniencepanel interface 110 may be the same as, or different from, the marketfor which probabilistic panel data is obtained by the probabilisticpanel interface 105. For example, the market for the convenience paneldata may be a first, target market of interest for which audiencemeasurement data is being determined, whereas the market for theprobabilistic panel data may be a second, calibration market differentfrom the first market, which is characterized as having better qualitypanel data than is available in the first market.

In the illustrated example, the behavioral distribution estimator 115 isstructured to determine respective behavioral universe estimate (UE)distributions for the probabilistic panel data obtained by theprobabilistic panel interface 105 and for the convenience panel dataobtained by the convenience panel interface 110. In some examples, abehavioral UE represents a total number of behavior-related events in atarget population. In some such examples, a marginal behavioral UErepresents a fraction of the total number of behavior-related events inthe target population that are associated with a set of one or morepanelist, behavior-related characteristics (or categories) associatedwith a given margin. In some such examples, a behavioral UE distributionrepresents the collection of marginal behavioral UEs for the giventarget population. In some such examples, a behavioral UE distributioncorresponds to a UE distribution in which the panelistcharacteristics/categories of the UE margins are based on panelistbehaviors relative to accessing the online media (e.g., such asnumber/frequency of website visits in a measurement period,number/frequency of online advertisements accessed in a measurementperiod, number/frequency of online media (e.g., programs, movies, clips,etc.) accessed in a measurement period, etc.). This is in contrast witha demographic UE distribution, which corresponds to a UE distribution inwhich UE and marginal UEs represent numbers of individuals (rather thanevents) in the population, and the characteristics/categories of the UEmargins are based on panelist demographics (e.g., such as age, gender,income, etc.).

An example of a behavioral UE distribution determined by the examplebehavioral distribution estimator 115 for probabilistic panel dataobtained by the probabilistic panel interface 105 is illustrated inTable 1. In Table 1, the behavioral UE distribution represents the totalcounts of panelists who visited search engines/portals and communitiesin a given market (e.g., the U.S. market) in a given measurement period(e.g., 1 month) and in different behavioral categories corresponding todifferent frequencies of accessing websites in the given measurementperiod. Thus, in the example of Table 1, the behavioral distributionestimator 115 is configured to determine a probabilistic panelbehavioral UE distribution in which the marginal behavioral UEs of thedistribution represent the sizes of different groups of panelists havingdifferent frequencies with which the panelists access the searchengines/portals and communities in the given market. For example, thebehavioral distribution estimator 115 may rank the website visits fordifferent panelists in order of the frequencies with which the differentpanelists access the monitored websites over the measurement period andthen divide the ranked website visits into quartiles (e.g., fourths). Inthe example of Table 1, the 1^(st) Quartile corresponds to the number ofpanelists falling into the lowest quartile of website visit frequency,whereas the 4^(th) Quartile corresponds to the number of panelistsfalling into the highest quartile of website visit frequency. In theexample of Table 1, the marginal behavioral counts are also weightedbased on one or more demographic characteristics to represent thedemographics of the population. In the example of Table 1, thebehavioral distribution estimator 115 also determines marginalbehavioral counts representative of panelists that accessed websites inthe measurement period but not those monitored websites in the categoryof interest (corresponding to NO VISIT in Table 1) and panelists thatwere online but had no browser activity in the measurement period(corresponding to INACTIVE in Table 1). As shown in Table 1, thebehavioral distribution estimator 115 can be configured to determine thebehavioral UE distribution as relative percentages (as shown in the lastcolumn of the table).

TABLE 1 Weighted Estimate Weighted Distribution Search Engines/ NO VISIT35539155 13.61% Portals & 1st QUARTILE 45870421 17.57% Communities 2ndQUARTILE 46043828 17.63% 3rd QUARTILE 45347714 17.37% 4th QUARTILE45515594 17.43% INACTIVE 42796402 16.39%

To implement multi-market calibration of convenience panel data toreduce behavioral bias, the behavioral distribution estimator 115 of theillustrated example determines a behavioral distribution, such as adistribution like the example illustrated in Table 1, for conveniencepanel data obtained for a first, target market of interest during afirst measurement interval. For example, the target market maycorrespond to a European Union (EU) country, and the first measurementinterval may correspond to a given month, such as the current month. Thebehavioral distribution estimator 115 of the illustrated example alsodetermines a behavioral distribution, such as a distribution like theexample illustrated in Table 1, for probabilistic panel data obtainedfor a second, calibration market during the first measurement interval.For example, the second market may correspond to the United States(U.S.). The calibration distribution selector 120 of the illustratedexample then selects the probabilistic panel behavioral distribution forthe second market to be an initial distribution (e.g., to be a seed) foruse in iteratively determining subsequent target behavioraldistributions for use in audience measurement in the first market.

The behavioral distribution calibrator 125 of the illustrated examplethen combines the initial calibration behavioral distribution (which isthe probabilistic panel behavioral distribution for the second market)with the convenience panel behavioral distribution determined by thebehavioral distribution estimator 115 for the target market to determinea new (posterior) target behavioral distribution for the first market tobe used to determine audience measurement data associated with the firstmeasurement period and the first market. For example, the behavioraldistribution calibrator 125 may combine the initial calibrationbehavioral distribution for the calibration market with the conveniencepanel behavioral distribution for the target market by weighting theinitial calibration behavioral distribution (e.g., by weighting theindividual marginal counts of the initial calibration behavioraldistribution) and weighting the convenience panel behavioraldistribution (e.g., by weighting the individual marginal counts of theconvenience panel behavioral distribution) and then combining theweighted distributions (e.g., by combining respective weighting marginalcounts from the two distributions) to determine a new (posterior) targetbehavioral distribution for the target market. In some examples, theweights can be set to be equal (e.g., such as to a value of 0.5) tocause the initial calibration behavioral distribution and theconvenience panel behavioral distribution to contribute equally to thecombination. In some examples, the weights for the initial calibrationbehavioral distribution may be different than the weights for theconvenience panel behavioral distribution to cause one distribution tocontribute more to the combination than the other distribution.

In the illustrated example of FIG. 1 , the target convenience panel datacalibrator 130 reports the new (posterior) target behavioral UEdistribution determined by the behavioral distribution calibrator 125for the target market and first measurement period to downstreamaudience measurement processing. In some examples, the targetconvenience panel data calibrator 130 may also use the new (posterior)target behavioral UE distribution determined for the target market andfirst measurement period to calibrate (e.g., weight) convenience paneldata obtained by the convenience panel interface 110 for the targetmarket during the first measurement period to reduce behavioral biaspresent in the convenience panel data obtained for the first market. Insome examples, the downstream audience measurement processing may usethe calibrated convenience panel data and/or the target behavioral UEdistribution reported by the target convenience panel data calibrator130 for the target market and first measurement period to determineaudience estimate(s) (e.g., ratings, reach, impressions, etc.) foronline media in the target market.

In the illustrated example, the target convenience panel data calibrator130 also provides the new (posterior) target behavioral distributiondetermined for the target market and first measurement period to thecalibration distribution selector 120, which selects that distributionto be the initial calibration behavioral distribution for the nextprocessing iteration corresponding to the next measurement period. Then,during the next processing iteration corresponding to the nextmeasurement period, the behavioral distribution estimator 115 of theillustrated example determines a convenience panel behavioraldistribution for the target market of interest and for the nextmeasurement period (e.g., the next month). The behavioral distributioncalibrator 125 then combines the target behavioral distribution from theprior measurement period and the convenience panel behavioraldistribution for the next measurement period to determine a new(posterior) target behavioral distribution for the next measurementperiod. This iterative processing continues with the new (posterior)target behavioral distribution determined during one iteration becomingthe initial calibration behavioral distribution to be used during thenext processing iteration. Thus, in the illustrated example of FIG. 1 ,the probabilistic panel behavioral distribution for the calibrationmarket serves as a probabilistic starting point for calibrating futuretarget behavioral distributions that are to be weighted and combinedwith convenience panel behavioral distributions associated with futuremeasurement periods in the target market to determine new targetbehavioral distributions to be used for audience measurementcalculations associated with those future measurement periods. However,the influence of the calibration market's data lessens with eachprocessing iteration.

As mentioned above, the behavioral distributions determined by thebehavioral distribution estimator 115 may be weighted based on one ormore demographic characteristics of a population, and the resulting new(posterior) target behavioral distributions determined by the audiencemeasurement system 100 may be used in downstream audience measurementprocessing. An example of such an overall audience measurement processis as follows:

1. Obtain demographic UEs for the target market of interest (e.g., thefirst market).

2. Weight the probabilistic panel data for the calibration market (e.g.,the second market) based on the demographic UEs.

3. Use the weighted probabilistic panel data as input for the firstprocessing iteration in the audience measurement system 100, asdescribed above.

4. Combine the convenience panel data for the target market withprobabilistic panel data for the target market (if the latter isavailable)

5. Weight the convenience panel data (or combined panel data determinedfrom step 4) for the target market based on the demographic UEs and thenew (posterior) target behavioral distribution determined by theaudience measurement system 100 for the current measurement period.

6. Determine and report audience measurement metrics based on theweighted, convenience panel data (or the weighted, combined panel data,if available) from step 5.

A block diagram of a second example audience measurement system 200implementing multi-market calibration of convenience panel data toreduce behavioral bias in accordance with teachings of this disclosureis illustrated in FIG. 2 . The example audience measurement system 200is structured to calibrate convenience panel data (also referred toherein as a convenience sample) obtained for a target market by creatingbehavioral targets for the target market of interest (also referred toas market A in the following description) using (i) geo-demographicallyweighted convenience panel data from market A and (ii) a relationship ofa reweighted probabilistic panel behavioral distribution (also referredto herein as a probabilistic sample behavioral distribution) to areweighted convenience panel behavioral distribution (also referred toherein as a convenience sample behavioral distribution) in one or morecalibration markets (also referred to as markets B, C, . . . , in thefollowing description). For example, market A may correspond to a targetcountry of interest for which convenience panel data is to becalibrated, market B may correspond to a first calibration market, suchas a first calibration country, to be used to calibrate the conveniencepanel data for market A, and markets C, D, E, . . . , may correspond toother calibration markets, such as other calibration countries, to beused to calibrate the convenience panel data for market A. The exampleaudience measurement system 200 then weights the convenience panel datain market A based on existing geodemographic controls and the behavioraltargets determined via the aforementioned calibration process to obtaincalibrated convenience panel data in market A, which can be used todetermine online media audience estimate(s) (e.g., ratings, reach,impressions, etc.) with reduced behavioral bias relative to uncalibratedconvenience panel data.

Turing to FIG. 2 , similar to the example audience measurement system100 of FIG. 1 , the example audience measurement system 200 includes theexample probabilistic panel interface 105, the example convenience panelinterface 110, an example behavioral distribution estimator 215, anexample behavioral distribution calibrator 225 and an example targetconvenience panel data calibrator 230. In the illustrated example ofFIG. 2 , the probabilistic panel interface 105 is structured, asdescribed above, to interface (e.g., via one or more networks) with oneor more example calibration market probabilistic panel data source(s)205. In the illustrated example, the calibration market probabilisticpanel data source(s) 205 provide probabilistic panel data for one ormore calibration markets, which are different from the target market forwhich convenience panel data is to be calibrated. For example, theprobabilistic panel interface 105 may interface with one or moreimpression monitoring system(s) that include metering devices and/ormonitoring applications that interface with and/or execute on computingdevices of probabilistic panelists in the one or more calibrationmarkets to generate online media impression data. Additionally oralternatively, in some examples, the probabilistic panel interface 105may interface (e.g., via one or more networks) with the metering devicesand/or monitoring applications that interface with and/or execute oncomputing devices of the probabilistic panelists in the one or morecalibration markets to obtain the online media impression data. Examplesof impression data are described above.

In the illustrated example of FIG. 2 , the convenience panel interface110 is structured, as described above, to interface (e.g., via one ormore networks) with one or more example target market convenience paneldata source(s) 210 and one or more example calibration marketconvenience panel data source(s) 212. In the illustrated example, thetarget market convenience panel data source(s) 210 provide conveniencepanel data for a given target market of interest for which conveniencepanel data is to be calibrated. For example, the convenience panelinterface 110 may interface with one or more impression monitoringsystem(s) that interface with websites serving online media in thetarget market to obtain online media impression data provided by thosewebsites (e.g., which is obtained from users in the target market inresponse to prompts from the websites when the users access online mediafrom the websites). Additionally or alternatively, in some examples, theconvenience panel interface 110 may interface (e.g., via one or morenetworks) with the websites serving online media in the target market toobtain the online media impression data from the websites. Similarly,the calibration market convenience panel data source(s) 212 provideconvenience panel data for one or more calibration markets, which aredifferent from the target market for which convenience panel data is tobe calibrated. For example, the convenience panel interface 110 mayinterface with one or more impression monitoring system(s) thatinterface with websites serving online media in the conveniencemarket(s) to obtain online media impression data provided by thosewebsites (e.g., which is obtained from users in the conveniencemarket(s) in response to prompts from the websites when the users accessonline media from the websites). Additionally or alternatively, in someexamples, the convenience panel interface 110 may interface (e.g., viaone or more networks) with the websites serving online media in theconvenience market(s) to obtain the online media impression data fromthe websites. Examples of impression data are described above.

The example audience measurement system 200 includes the examplebehavioral distribution estimator 215, which may be similar to theexample behavioral distribution estimator 115 of FIG. 1 . In theillustrated example of FIG. 2 , the example behavioral distributionestimator 215 is structured to determine behavioral distribution(s) forthe probabilistic panel data obtained by the probabilistic panelinterface 105 for the calibration market(s) (e.g., obtained from thecalibration market probabilistic panel data source(s) 205). Thebehavioral distribution estimator 215 is also structured to determinerespective behavioral distributions for the convenience panel dataobtained by the convenience panel interface 110 for the calibrationmarket(s) (e.g., obtained from the calibration market convenience paneldata source(s) 212) and for the target market (e.g., obtained from thetarget market convenience panel data source(s) 210). An exampleimplementation of the behavioral distribution estimator 215 of FIG. 2 isillustrated in FIG. 3 .

Turning to FIG. 3 , the behavioral distribution estimator 215 of theillustrated example includes an example behavioral distributioninitializer 305, an example target market panel data weighter 310, anexample calibration market panel data weighter 315, and an examplebehavioral distribution calculator 320. The behavioral distributioninitializer 305 is structured to define behavioral control variables,also referred to as behavioral categories, for which calibration targets(e.g., corresponding to a calibrated convenience panel distribution) areto be determined for the target market, referred to as market A in thefollowing description. By way of example, the behavioral categoriesdefined by the behavioral distribution initializer 305 could correspondto the behavioral categories specified in Table 1, which correspond todifferent panelist quartiles associated with different frequencies withwhich panelists in market A access websites being monitored. In someexamples, the behavioral distribution initializer 305 defines thebehavioral categories for which calibration targets are to be determinedbased on user input, one or more configuration scripts, etc.

In the illustrated example, the behavioral distribution initializer 305is also structured to obtain geodemographic weighting controls to beused to initially weight the convenience panel data for target market Abefore behavioral-based calibration is performed. For example, thebehavioral distribution initializer 305 may obtain demographic weightingcontrols that define one or more demographic targets that theconvenience panel data for target market A is to be weighted to meet.For example, the demographic targets can correspond to target agedistributions, gender distributions, income distributions, educationlevel distributions, etc., or any combination thereof, representative ofthe population in market A. Additionally or alternatively, thebehavioral distribution initializer 305 may obtain location weightingcontrols that define one or more location targets that the conveniencepanel data for target market A is to be weighted to meet. For example,the location targets can correspond to target population sizes indifferent geographic regions of market A. In some examples, thebehavioral distribution initializer 305 obtains the demographicweighting controls and/or the location weighting controls from one ormore data sources, such as governmental population census source(s),market research source(s), etc.

In the illustrated example, the target market panel data weighter 310weights the convenience sample (or, in other words, the conveniencepanel data) for target market A to correspond to the geodemographicweighting controls obtained by the behavioral distribution initializer305 for target market A. For example, convenience sample for targetmarket A may include demographic and/or location data associated withthe impression data included in the panelist entries represented in theconvenience sample. In some such examples, the target market panel dataweighter 310 may scale, replicate, remove, etc., ones of the panelistentries included in the convenience sample for target market A such thatthe resulting contributions of impressions for different demographicand/or location categories correspond to the geodemographic weightingcontrols obtained by the behavioral distribution initializer 305 fortarget market A. By way of example, the target market panel dataweighter 310 may weight (e.g., scale, replicate and/or remove) panelistentries in the convenience sample for target market A that correspond toa first demographic category to match a demographic weighting controlobtained by the behavioral distribution initializer 305 for that firstdemographic category in market A. Similarly, the market panel dataweighter 310 may weight (e.g., scale, replicate and/or remove) panelistentries in the convenience sample for target market A that correspond toa second demographic category to match a demographic weighting controlobtained by the behavioral distribution initializer 305 for that seconddemographic category in market A, and so on for different demographicweighting controls obtained by the behavioral distribution initializer305 for market A. Additionally or alternatively, the market panel dataweighter 310 may perform similar weighting for different locationcategories represented in the convenience sample based on respectivelocation weighting controls obtained by the behavioral distributioninitializer 305.

In the following description, the resulting weighted convenience sampledetermined by the target market panel data weighter 310 for targetmarket A is referred to as the target market convenience sample A2.

In the illustrated example, the calibration market panel data weighter315 weights the probabilistic sample (or, in other words, theprobabilistic panel data) for calibration market B to correspond to thedemographic weighting controls obtained by the behavioral distributioninitializer 305 for target market A. For example, probabilistic samplefor calibration market B may include demographic data associated withthe impression data included in the panelist entries represented in theprobabilistic sample. In some such examples, the calibration marketpanel data weighter 315 may scale, replicate, remove, etc., ones of thepanelist entries included in the probabilistic sample for calibrationmarket B such that the resulting contributions of impressions fordifferent demographic categories correspond to the demographic weightingcontrols obtained by the behavioral distribution initializer 305 fortarget market A. By way of example, the calibration market panel dataweighter 315 may weight (e.g., scale, replicate and/or remove) panelistentries in the probabilistic sample for calibration market B thatcorrespond to a first demographic category to match the weightingcontrol obtained by the behavioral distribution initializer 305 for thatdemographic category in target market A. Similarly, the calibrationmarket panel data weighter 315 may weight (e.g., scale, replicate and/orremove) panelist entries in the probabilistic sample for calibrationmarket B that correspond to a second demographic category to match theweighting control obtained by the behavioral distribution initializer305 for that demographic category in market A, and so on for differentdemographic categories obtained by the behavioral distributioninitializer 305 for market A.

In some examples, the probabilistic sample for calibration market B maybe geographically disproportionate relative to the convenience samplefor calibration market B and/or relative to the universe of calibrationmarket B. For example, different geographic regions in calibrationmarket B may contribute differently to the probabilistic sample and/orconvenience sample based on different panel sampling rates used in therespective geographic regions. For example, a first geographic region incalibration market B may contain 50% of the probabilistic sample incalibration market B, whereas two other geographic regions contain 20%and 30%, respectively, of the probabilistic sample. The conveniencesample in calibration market B may contribute 40%, 30% and 30% from therespective regions. Thus, in such an example, the probabilistic samplefrom calibration market B is distributed geographically differently thanthe convenience sample in the first and second geographic regions.

In some examples, to account for a geographically disproportionateprobabilistic and convenience samples from calibration market B, thecalibration market panel data weighter 315 applies geographic weightingto the probabilistic sample from market B. For example, the calibrationmarket panel data weighter 315 can determine geographic weightingtargets to apply to the probabilistic sample from calibration market Bby determining a geography UE distribution for the probabilistic samplefrom market B, and then applying that geography UE distribution formarket B to the aggregate universe (e.g., sample size) of theconvenience sample from the target market A. In some examples, themarket panel data weighter 315 determines the geography UE distributionfor the probabilistic sample from market B in a manner similar to howthe behavioral UE distribution are determined, but with the distributioncategories corresponding to different geographic locations/regionsinstead of different behavior categories.

In the following description, the resulting weighted probabilisticsample determined by the calibration market panel data weighter 315 forcalibration market B is referred to as the calibration marketprobabilistic sample B1.

In the illustrated example, the calibration market panel data weighter315 similarly weights the convenience sample (or, in other words, theconvenience panel data) for calibration market B to correspond to thedemographic weighting controls obtained by the behavioral distributioninitializer 305 for target market A. For example, convenience sample forcalibration market B may include demographic associated with theimpression data included in the panelist entries represented in theconvenience sample. In some such examples, the calibration market paneldata weighter 315 may scale, replicate, remove, etc., ones of thepanelist entries included in the convenience sample for calibrationmarket B such that the resulting contribution of impressions fordifferent demographic categories correspond to the demographic weightingcontrols obtained by the behavioral distribution initializer 305 fortarget market A. By way of example, the calibration market panel dataweighter 315 may weight (e.g., scale, replicate and/or remove) panelistentries in the convenience sample for calibration market B thatcorrespond to a first demographic category to match the weightingcontrol obtained by the behavioral distribution initializer 305 for thatdemographic category in market A. Similarly, the calibration marketpanel data weighter 315 may weight (e.g., scale, replicate and/orremove) panelist entries in the convenience sample for calibrationmarket B that correspond to a second demographic category to match theweighting control obtained by the behavioral distribution initializer305 for that demographic category in market A, and so on for differentdemographic categories obtained by the behavioral distributioninitializer 305 for market A.

In some examples, the convenience sample for calibration market B may begeographically disproportionate relative to the probabilistic sample forcalibration market B and/or relative to the universe of calibrationmarket B. For example, different geographic regions in calibrationmarket B may contribute differently to the convenience sample andprobability sample based on different panel sampling rates used in therespective geographic regions. In some examples, to account forgeographically disproportionate convenience and probability samples fromcalibration market B, the calibration market panel data weighter 315applies geographic weighting to the convenience sample from market B.For example, the calibration market panel data weighter 315 can use thegeographic weighting targets determined, as described above, forweighting the probabilistic sample from market B to also weight theconvenience sample from market B.

In the following description, the resulting weighted convenience sampledetermined by the calibration market panel data weighter 315 forcalibration market B is referred to as the calibration marketconvenience sample B2.

Next, the behavioral distribution calculator 320 uses the behavioralcontrol variables determined by the behavioral distribution initializer305 to initialize the behavioral categories for the behavioraldistributions to be determined for the target market convenience sampleA2, the calibration market probabilistic sample B1 and the calibrationmarket convenience sample B2. In some examples, the behavioraldistribution calculator 320 omits behavioral categories not included inthe calibration market probabilistic sample B1. For each behavioralcategory, the behavioral distribution calculator 320 computes therespective percentage of the weighted calibration market probabilisticsample B1 in each defined behavioral category to determine thebehavioral distribution for the calibration market probabilistic sampleB1. In the following description, the behavioral distribution percentagevalue of the c^(th) behavioral category in the behavioral distributionfor the calibration market probabilistic sample B1 is denoted by B1 c%.

Likewise, for each behavioral category, the behavioral distributioncalculator 320 computes the respective percentage of the weightedcalibration market convenience sample B2 in each defined behavioralcategory to determine the behavioral distribution for the calibrationmarket convenience sample B2. In the following description, thebehavioral distribution percentage value of the c^(th) behavioralcategory in the behavioral distribution for the calibration marketconvenience sample B2 is denoted by B2 c%. Likewise, for each behavioralcategory, the behavioral distribution calculator 320 computes therespective percentage of the weighted target market convenience sampleA2 in each defined behavioral category to determine the behavioraldistribution for the target market convenience sample A2. In thefollowing description, the behavioral distribution percentage value ofthe c^(th) behavioral category in the behavioral distribution for thetarget market convenience sample A2 is denoted by A2 c%.

Returning to FIG. 2 , the behavioral distribution estimator 215 outputsthe behavioral distribution for the target market convenience sample A2(e.g., the values of A2 c% for the different behavioral categories c),the behavioral distribution for the calibration market probabilisticsample B1 (e.g., the values of B1 c% for the different behavioralcategories c), and the behavioral distribution for the calibrationmarket convenience sample B2 (e.g., the values of B2 c% for thedifferent behavioral categories c) to the behavioral distributioncalibrator 225. In the illustrated example, the behavioral distributioncalibrator 225 calibrates the behavioral distribution for the targetmarket convenience sample A2 (e.g., the values of A2 c% for thedifferent behavioral categories c) based on the behavioral distributionfor the calibration market probabilistic sample B1 (e.g., the values ofB1 c% for the different behavioral categories c) and the behavioraldistribution for the calibration market convenience sample B2 (e.g., thevalues of B2 c% for the different behavioral categories c). An exampleimplementation of the behavioral distribution calibrator 225 isillustrated in FIG. 4 .

Turning to FIG. 4 , the behavioral distribution calibrator 225 of theillustrated example includes an example preliminary distributioncalibrator 405 and an example final distribution calibrator 410. Thepreliminary distribution calibrator 405 performs an initial calibrationof the behavioral distribution for the target market convenience sampleA2 based on the behavioral distribution for the calibration marketprobabilistic sample B1 and the behavioral distribution for thecalibration market convenience sample. For example, for each behavioralcategory c, the preliminary distribution calibrator 405 computes apreliminary calibrated value for the value A2 c% of the behavioraldistribution for the target market convenience sample A2 using the valueB1 c% of the behavioral distribution for the calibration marketprobabilistic sample B1 and the value B2 c% of the behavioraldistribution for the calibration market convenience sample B2 accordingto Equation 1, which is:

PTBc=(B 1 c%/B 2 c%)×A 2 c%   Equation 1

Thus, according to Equation 1, for each behavioral category c, thepreliminary distribution calibrator 405 scales the value A2 c% of thebehavioral distribution for the target market convenience sample A2 by aratio of (i) the value B1 c% of the behavioral distribution for thecalibration market probabilistic sample B1 to (ii) the value B2 c% ofthe behavioral distribution for the calibration market conveniencesample B2 to determine a preliminary calibrated value for the behavioraldistribution for the target market convenience sample A2. Thispreliminary calibrated value is denoted PTBc in Equation 1.

In some examples, if the value B1 c% for the c ^(th) category of thebehavioral distribution for the calibration market probabilistic sampleB1 does satisfy a threshold (e.g., is less than the threshold), then aratio of (i) the value of a broader (or upper-level, or higher-level)hierarchical behavior category in the behavioral distribution for thecalibration market probabilistic sample B1 to (ii) the value of thebroader hierarchical behavior category in the behavioral distributionfor the calibration market convenience sample B2 is used to scale thevalue A2 c% of the behavioral distribution for the target marketconvenience sample A2. For example, the threshold may be configurablebased on user input, specified as a configuration parameter, hard-coded,etc. In some examples, the behavioral categories of a behavioraldistribution may be arranged in a hierarchical fashion. For example, afirst, higher level of a behavioral category hierarchy may segment thepanel according to types of online media access, with the first levelcategories being, for example, news, entertainment, public service andcommercials. A second, lower level of the behavioral category hierarchymay segment the behavioral category of news media access into politicalnews, sports news and weather news. If the preliminary distributioncalibrator 405 is attempting to calibrate one of the lower levelcategories (e.g., political news, sports news or weather news) of thebehavioral distribution for the target market convenience sample A2, andthat category in the behavioral distribution for the calibration marketprobabilistic sample B1 does not satisfy the threshold, the preliminarydistribution calibrator 405 uses the higher level category correspondingto the lower level category (e.g., news in this example) in thebehavioral distribution for the calibration market probabilistic sampleB1 and in the behavioral distribution for the calibration market targetsample B2 to calibrate that lower level behavioral category in thebehavioral distribution for the target market convenience sample A2.

For example, assume the value A2 c% of the behavioral distribution forthe target market convenience sample A2 corresponds to the “politicalnews” behavioral category in the preceding example, and thecorresponding value B1 c% of the “political news” behavioral category inthe behavioral distribution for the calibration market probabilisticsample B1 does not satisfy the threshold. Also assume that therespective values of the higher-level category “news” in the behavioraldistribution for the calibration market probabilistic sample B1 and inthe behavioral distribution for the calibration market target sample B2are B1 c′% and B2 c′%, respectively. Then, the preliminary distributioncalibrator 405 computes a preliminary calibrated value for the value A2c% of the behavioral distribution for the target market conveniencesample A2 using the value B1 c′% of the behavioral distribution for thecalibration market probabilistic sample B1 and the value B2 c′% of thebehavioral distribution for the calibration market convenience sample B2according to Equation 2, which is:

PTBc=(B 1 c′%/B 2 c′%)×A 2 c%   Equation 2

The final distribution calibrator 410 performs a final calibration ofthe behavioral distribution for the target market convenience sample A2to normalize the values of the behavioral distribution to sum to 100%(or a probabilistic value of 1). In some examples, the preliminarycalibrated values, PTBc, determined by the preliminary distributioncalibrator 405 for the behavioral distribution for the target marketconvenience sample A2 do not sum to 100% (or a probabilistic value of1). Thus, for each behavioral category c, the final distributioncalibrator 410 computes a final calibrated value, denoted FTBc, for thepreliminary calibrated value, denoted PTBc, of the behavioraldistribution for the target market convenience sample A2 according toEquation 3, which is:

FTBc=(100/ΣcPTBc)×PTBc.   Equation 3.

In Equation 3, the summation is over the behavioral categories definedfor the behavioral distribution for the target market convenience sampleA2.

In some examples, the final distribution calibrator 410 outputs thevalues of FTBc as the calibrated behavioral distribution for the targetmarket convenience sample A2. For example, such values of FTBc output bythe final distribution calibrator 410 can correspond to calibratedversions of the values in the “Weighted Distribution” column of Table 1above.

In some examples, the behavioral distribution estimator 215 and thebehavioral distribution calibrator 225 of the audience measurementsystem 200 repeat the foregoing operations for additional calibrationmarkets (e.g., such as calibration markets C, D, E, etc.). In suchexamples, the behavioral distribution estimator 215 and the behavioraldistribution calibrator 225 determine different calibrated behavioraldistributions for the target market convenience sample A2, which eachone being calibrated based on a different one of the calibrationmarkets. However, in some such examples, the behavioral categoriesinitialized for some, or all, of the behavioral distributions determinedby the behavioral distribution estimator 215 are limited to thebehavioral categories included in the included in the calibration samplefor the first calibration market, such as market B in the precedingexamples.

In some examples, if additional calibration markets are utilized, thebehavioral distribution calibrator 225 computes the final calibratedbehavioral distribution for the target market convenience sample A2 asthe average the calibrated behavioral distributions determined based onthe different calibration markets. In some examples, the behavioraldistribution calibrator 225 computes the final calibrated behavioraldistribution for the target market convenience sample A2 as a weightedaverage of the calibrated behavioral distributions determined based onthe different calibration markets, with some calibrated behavioraldistributions weighted more than others in the average based on one ormore criteria.

Returning to FIG. 2 , the target convenience panel data calibrator 230of the audience measurement reports the calibrated behavioral UEdistribution determined by the behavioral distribution calibrator 225for the convenience sample target market A and the given measurementperiod to downstream audience measurement processing. In some examples,the target convenience panel data calibrator 230 may also use thecalibrated behavioral UE distribution determined for the target market Aand the given measurement period to calibrate (e.g., weight) theconvenience panel data (e.g., convenience sample) obtained by theconvenience panel interface 110 for the target market A during the givenmeasurement period to reduce behavioral bias present in the conveniencepanel data obtained for the target market. For example, the targetconvenience panel data calibrator 230 may weight the convenience paneldata (e.g., convenience sample) obtained by the convenience panelinterface 110 for the target market A during the given measurementperiod by the geodemographic controls described above to meetdemographic and geographic-based targets, as well as by the calibratedbehavioral targets represented by the calibrated behavioral UEdistribution. The result is calibrated convenience panel data (e.g., thecalibrated convenience sample) for target market A during the givenmeasurement period. Additionally or alternatively, the targetconvenience panel data calibrator 230 may weight any other data sample,which is intended to represent the population universe of the targetmarket A and the given measurement period, by the geodemographiccontrols for the target market A and the calibrated behavioral targetsrepresented by the calibrated behavioral UE distribution to determine acorresponding, calibrated data sample for the target market A. In someexamples, the downstream audience measurement processing may use thecalibrated convenience panel data, any other calibrated data sample(s)determined for the target market, and/or the calibrated behavioral UEdistribution reported by the target convenience panel data calibrator230 for the target market and given measurement period to determineaudience estimate(s) (e.g., ratings, reach, impressions, etc.) foronline media in the target market. For example, such an output audienceestimate can correspond to calibrated versions of the values in the“Weighted Estimate” column of Table 1 above.

In the illustrated example of FIG. 2 , the probabilistic panel interface105, the convenience panel interface 110, the behavioral distributionestimator 215, the behavioral distribution calibrator 225 and theexample target convenience panel data calibrator 230 repeat theprocedure described above for each subsequent measurement interval todetermine a new, calibrated behavioral UE distribution for theconvenience sample of the target market for each subsequent measurementinterval. In some examples, the target convenience panel data calibrator230 also determines a calibrated convenience sample (and/or othercalibrated data samples(s)) for the target market and for eachsubsequent measurement interval using the calibrated behavioral UEdistribution determined for that subsequent measurement interval.

FIGS. 5A-5D illustrate a first example technique 500 for calibratingconvenience panel behavioral UE distributions for a first market basedon behavioral UE distributions determined for probabilistic panel dataassociated with the first market and behavioral UE distributionsdetermined for convenience panel data also associated with the firstmarket. The first example technique 500, which is different from theexamples described above in connection with FIGS. 1-4 , begins in afirst measurement period (e.g., Month 1) with an example conveniencepanel behavioral UE distribution 505 for the first market (e.g., thetarget market) and the first measurement period being weighted to alignwith an example probabilistic panel behavioral UE distribution 510 forthe same first market and the first measurement period to determine anexample calibrated convenience panel behavioral UE distribution 515 forthe first market and the first measurement period. Then, during a secondmeasurement period (e.g., Month 2), an example convenience panelbehavioral UE distribution 520 for the first market and the secondmeasurement period is weighted to align with an example probabilisticpanel behavioral UE distribution 525 for the same first market and thesecond measurement period to determine an example calibrated conveniencepanel behavioral UE distribution 530 for the first market and the secondmeasurement period. Then, during a third measurement period (e.g., Month3), an example convenience panel behavioral UE distribution 535 for thefirst market and the third measurement period is weighted to align withan example probabilistic panel behavioral UE distribution 540 for thefirst market and the third measurement period to determine an examplecalibrated convenience panel behavioral UE distribution 545 for thefirst market and the third measurement period. Then, during a fourthmeasurement period (e.g., Month 4), an example convenience panelbehavioral UE distribution 550 for the first market and the fourthmeasurement period is weighted to align with an example probabilisticpanel behavioral UE distribution 555 for the first market and the fourthmeasurement period to determine an example calibrated convenience panelbehavioral UE distribution 560 for the first market and the fourthmeasurement period. Thus, in the illustrated example of FIGS. 5A-5D, foreach measurement period, the convenience panel behavioral UEdistribution obtained for the first market and for that measurementperiod is weighted to align with the probabilistic panel behavioral UEdistribution obtained for that same first market and measurement period.

FIGS. 6A-6D illustrate a second example technique 600 for determiningbehavioral UE distributions for a first market based on behavioral UEdistributions determined for just convenience panel data associated withthe first market. The second example technique 600, which is differentfrom the examples described above in connection with FIGS. 1-4 , beginsin a first measurement period (e.g., Month 1) with an exampleconvenience panel behavioral UE distribution 605 for the first market(e.g., target market) and the first measurement period being usedwithout probabilistic panel data to determine an example targetbehavioral UE distribution 610 for the first market and the firstmeasurement period. Then, during a second measurement period (e.g.,Month 2), an example convenience panel behavioral UE distribution 615for the first market and the second measurement period is used withoutprobabilistic panel data to determine an example target behavioral UEdistribution 620 for the first market and the second measurement period.Then, during a third measurement period (e.g., Month 3), an exampleconvenience panel behavioral UE distribution 625 for the first marketand the third measurement period is used without probabilistic paneldata to determine an example target behavioral UE distribution 630 forthe first market and the third measurement period. Then, during a fourthmeasurement period (e.g., Month 4), an example convenience panelbehavioral UE distribution 635 for the first market and the fourthmeasurement period is used without probabilistic panel data to determinean example target behavioral UE distribution 640 for the first marketand the fourth measurement period.

FIGS. 7A-7D illustrate a third example technique 700 for calibratingconvenience panel behavioral UE distributions for a first market (e.g.,the target market) iteratively based on an initial behavioral UEdistribution determined from probabilistic panel data associated with asecond market (e.g., the calibration market) different from the firstmarket. The third example technique 700, which corresponds to theexample described above in connection with FIG. 1 , begins in a firstmeasurement period (e.g., Month 1) with an example convenience panelbehavioral UE distribution 705 for the first market and the firstmeasurement period being weighted and combined with an example weightedprobabilistic panel behavioral UE distribution 710 for the second marketand the first measurement period to determine an example calibratedbehavioral UE distribution 715 for the first market and the firstmeasurement period. Then, during a second measurement period (e.g.,Month 2), an example convenience panel behavioral UE distribution 720for the first market and the second measurement period is weighted andcombined with the weighted, calibrated behavioral UE distribution 715determined for the first measurement period to determine a new examplecalibrated behavioral UE distribution 730 for the first market and thesecond measurement period. Then, during a third measurement period(e.g., Month 3), an example convenience panel behavioral UE distribution735 for the first market and the third measurement period is weightedand combined with the weighted, calibrated behavioral UE distribution730 determined for the second measurement period to determine a newexample calibrated behavioral UE distribution 745 for the first marketand the third measurement period. Then, during a fourth measurementperiod (e.g., Month 4), an example convenience panel behavioral UEdistribution 750 for the first market and the fourth measurement periodis weighted and combined with the weighted, calibrated behavioral UEdistribution 745 determined for the third measurement period todetermine a new example calibrated behavioral UE distribution 460 forthe first market and the fourth measurement period. Thus, in theillustrated example of FIGS. 7A-7D, for an initial measurement period,the convenience panel behavioral UE distribution obtained for the firstmarket and for that measurement period is weighted and combined with aweighted, probabilistic panel behavioral UE distribution obtained for adifferent second market to determine a calibrated behavioral UEdistribution for the first market and the initial measurement period.Thereafter, for each subsequent measurement period, the conveniencepanel behavioral UE distribution obtained for the first market and for agiven subsequent measurement period is weighted and combined with theweighted, calibrated behavioral UE distribution determined for the priormeasurement period to determine a new calibrated behavioral UEdistribution for the first market and the given measurement period.

FIGS. 8A-8D illustrate a fourth example technique 800 for calibratingbehavioral UE distributions determined from convenience panel data for afirst market (e.g., the target market) based on behavioral UEdistributions determined from probabilistic panel data and conveniencepanel data associated with a second market (e.g., the calibrationmarket), which is different from the first market. The fourth exampletechnique 800, which corresponds to the example described above inconnection with FIGS. 2-4 , begins in a first measurement period (e.g.,Month 1) with an example convenience panel behavioral UE distribution805 for the first market and the first measurement period beingcalibrated based on an example combination 810 of probabilistic paneland convenience panel behavioral UE distributions for the second marketand the first measurement period to determine an example calibratedconvenience panel behavioral UE distribution 815 for the first marketand the first measurement period. Then, during a second measurementperiod (e.g., Month 2), an example convenience panel behavioral UEdistribution 820 for the first market and the second measurement periodis calibrated based on an example combination 825 of probabilistic paneland convenience panel behavioral UE distributions for the second marketand the second measurement period to determine a new example calibratedconvenience panel behavioral UE distribution 830 for the first marketand the second measurement period. Then, during a third measurementperiod (e.g., Month 3), an example convenience panel behavioral UEdistribution 835 for the first market and the third measurement periodis calibrated based on an example combination 840 of probabilistic paneland convenience panel behavioral UE distributions for the second marketand the third measurement period to determine a new example calibratedconvenience panel behavioral UE distribution 845 for the first marketand the third measurement period. Then, during a fourth measurementperiod (e.g., Month 4), an example convenience panel behavioral UEdistribution 850 for the first market and the fourth measurement periodis calibrated based on an example combination 855 of probabilistic paneland convenience panel behavioral UE distributions for the second marketand the fourth measurement period to determine a new example calibratedconvenience panel behavioral UE distribution 860 for the first marketand the fourth measurement period. Thus, in the illustrated example ofFIGS. 7A-7D, for each measurement period, the convenience panelbehavioral UE distribution obtained for the first market (e.g., targetmarket) and for that measurement period is calibrated based on acombination of probabilistic panel and convenience panel behavioral UEdistributions obtained for a different second market to determine acalibrated convenience panel behavioral UE distribution for the firstmarket and that measurement period.

FIGS. 9A-9C depict a comparison of the first example technique 500illustrated in FIGS. 5A-5D, the second example technique 600 illustratedin FIGS. 6A-6D, and the fourth example technique 800 illustrated inFIGS. 8A-8D. In particular, FIG. 9A depicts the resulting calibratedbehavioral UE distributions 515, 530, 545 and 560 determined by thefirst example technique 500 during the four example measurement periods,as described above in connection with FIGS. 5A-5D. FIG. 9B depicts theresulting behavioral UE distributions 610, 620, 630 and 640 determinedby the second example technique 600 during the four example measurementperiods, as described above in connection with FIGS. 6A-6D. FIG. 9Cdepicts the resulting calibrated behavioral UE distributions 815, 830,845 and 860 determined by the fourth example technique 800 during thefour example measurement periods, as described above in connection withFIGS. 8A-8D. As can be seen from FIGS. 9A-9C, the fourth exampletechnique 800, which can be implemented by the example audiencemeasurement system 200 of FIGS. 2-4 , yields calibrated behavioral UEdistributions with less volatility than the behavioral UE distributionsobtained from the other example techniques 500 and 600.

While example manners of implementing the audience measurement system100 and the audience measurement system 100 are illustrated in FIGS. 1-4, one or more of the elements, processes and/or devices illustrated inFIGS. 1-4 may be combined, divided, re-arranged, omitted, eliminatedand/or implemented in any other way. Further, the example probabilisticpanel interface 105, the example convenience panel interface 110, theexample behavioral distribution estimator 115, the example calibrationdistribution selector 120, the example behavioral distributioncalibrator 125, the example convenience panel data calibrator 130, theexample behavioral distribution estimator 215, the example behavioraldistribution calibrator 225, the example target convenience panel datacalibrator 230, the example behavioral distribution initializer 305, theexample target market panel data weighter 310, the example calibrationmarket panel data weighter 315, the example behavioral distributioncalculator 320, the example preliminary distribution calibrator 405, theexample final distribution calibrator 410 and, more generally, theexample audience measurement system 100 and/or the audience measurementsystem 200 of FIGS. 1-4 may be implemented by hardware, software,firmware and/or any combination of hardware, software and/or firmware.Thus, for example, any of the example probabilistic panel interface 105,the example convenience panel interface 110, the example behavioraldistribution estimator 115, the example calibration distributionselector 120, the example behavioral distribution calibrator 125, theexample convenience panel data calibrator 130, the example behavioraldistribution estimator 215, the example behavioral distributioncalibrator 225, the example target convenience panel data calibrator230, the example behavioral distribution initializer 305, the exampletarget market panel data weighter 310, the example calibration marketpanel data weighter 315, the example behavioral distribution calculator320, the example preliminary distribution calibrator 405, the examplefinal distribution calibrator 410 and, more generally, the exampleaudience measurement system 100 and/or the example audience measurementsystem 200 could be implemented by one or more analog or digitalcircuit(s), logic circuits, programmable processor(s), programmablecontroller(s), graphics processing unit(s) (GPU(s)), digital signalprocessor(s) (DSP(s)), application specific integrated circuit(s)(ASIC(s)), programmable logic device(s) (PLD(s)), field programmablegate arrays (FPGAs) and/or field programmable logic device(s) (FPLD(s)).When reading any of the apparatus or system claims of this patent tocover a purely software and/or firmware implementation, at least one ofthe example audience measurement system 100, the example audiencemeasurement system 200, the example probabilistic panel interface 105,the example convenience panel interface 110, the example behavioraldistribution estimator 115, the example calibration distributionselector 120, the example behavioral distribution calibrator 125, theexample convenience panel data calibrator 130, the example behavioraldistribution estimator 215, the example behavioral distributioncalibrator 225, the example target convenience panel data calibrator230, the example behavioral distribution initializer 305, the exampletarget market panel data weighter 310, the example calibration marketpanel data weighter 315, the example behavioral distribution calculator320, the example preliminary distribution calibrator 405 and/or theexample final distribution calibrator 410 is/are hereby expresslydefined to include a non-transitory computer readable storage device orstorage disk such as a memory, a digital versatile disk (DVD), a compactdisk (CD), a Blu-ray disk, etc. including the software and/or firmware.Further still, the example audience measurement system 100 and/or theexample audience measurement system 200 may include one or moreelements, processes and/or devices in addition to, or instead of, thoseillustrated in FIGS. 1-4 , and/or may include more than one of any orall of the illustrated elements, processes and devices. As used herein,the phrase “in communication,” including variations thereof, encompassesdirect communication and/or indirect communication through one or moreintermediary components, and does not require direct physical (e.g.,wired) communication and/or constant communication, but ratheradditionally includes selective communication at periodic intervals,scheduled intervals, aperiodic intervals, and/or one-time events.

Flowcharts representative of example hardware logic, machine readableinstructions, hardware implemented state machines, and/or anycombination thereof for implementing the audience measurement system 100and the audience measurement system 200 are shown in FIGS. 10-11 . Inthis example, the machine readable instructions may be one or moreexecutable programs or portion(s) thereof for execution by a computerprocessor, such as the processor 1212 shown in the example processorplatform 1200 discussed below in connection with FIG. 12 . The one ormore programs, or portion(s) thereof, may be embodied in software storedon a non-transitory computer readable storage medium such as a CD-ROM, afloppy disk, a hard drive, a DVD, a Blu-ray disk™, or a memoryassociated with the processor 1212, but the entire program or programsand/or parts thereof could alternatively be executed by a device otherthan the processor 1212 and/or embodied in firmware or dedicatedhardware. Further, although the example program(s) is(are) describedwith reference to the flowcharts illustrated in FIGS. 10-11 , many othermethods of implementing the example audience measurement system 100and/or the example audience measurement system 200 may alternatively beused. For example, with reference to the flowcharts illustrated in FIGS.10-11 , the order of execution of the blocks may be changed, and/or someof the blocks described may be changed, eliminated, combined and/orsubdivided into multiple blocks. Additionally or alternatively, any orall of the blocks may be implemented by one or more hardware circuits(e.g., discrete and/or integrated analog and/or digital circuitry, anFPGA, an ASIC, a comparator, an operational-amplifier (op-amp), a logiccircuit, etc.) structured to perform the corresponding operation withoutexecuting software or firmware.

The machine readable instructions described herein may be stored in oneor more of a compressed format, an encrypted format, a fragmentedformat, a packaged format, etc. Machine readable instructions asdescribed herein may be stored as data (e.g., portions of instructions,code, representations of code, etc.) that may be utilized to create,manufacture, and/or produce machine executable instructions. Forexample, the machine readable instructions may be fragmented and storedon one or more storage devices and/or computing devices (e.g., servers).The machine readable instructions may require one or more ofinstallation, modification, adaptation, updating, combining,supplementing, configuring, decryption, decompression, unpacking,distribution, reassignment, etc. in order to make them directly readableand/or executable by a computing device and/or other machine. Forexample, the machine readable instructions may be stored in multipleparts, which are individually compressed, encrypted, and stored onseparate computing devices, wherein the parts when decrypted,decompressed, and combined form a set of executable instructions thatimplement a program such as that described herein. In another example,the machine readable instructions may be stored in a state in which theymay be read by a computer, but require addition of a library (e.g., adynamic link library), a software development kit (SDK), an applicationprogramming interface (API), etc. in order to execute the instructionson a particular computing device or other device. In another example,the machine readable instructions may need to be configured (e.g.,settings stored, data input, network addresses recorded, etc.) beforethe machine readable instructions and/or the corresponding program(s)can be executed in whole or in part. Thus, the disclosed machinereadable instructions and/or corresponding program(s) are intended toencompass such machine readable instructions and/or program(s)regardless of the particular format or state of the machine readableinstructions and/or program(s) when stored or otherwise at rest or intransit.

As mentioned above, the example processes of FIGS. 10-11 may beimplemented using executable instructions (e.g., computer and/or machinereadable instructions) stored on a non-transitory computer and/ormachine readable medium such as a hard disk drive, a flash memory, aread-only memory, a compact disk, a digital versatile disk, a cache, arandom-access memory and/or any other storage device or storage disk inwhich information is stored for any duration (e.g., for extended timeperiods, permanently, for brief instances, for temporarily buffering,and/or for caching of the information). As used herein, the termnon-transitory computer readable medium is expressly defined to includeany type of computer readable storage device and/or storage disk and toexclude propagating signals and to exclude transmission media. Also, asused herein, the terms “computer readable” and “machine readable” areconsidered equivalent unless indicated otherwise.

“Including” and “comprising” (and all forms and tenses thereof) are usedherein to be open ended terms. Thus, whenever a claim employs any formof “include” or “comprise” (e.g., comprises, includes, comprising,including, having, etc.) as a preamble or within a claim recitation ofany kind, it is to be understood that additional elements, terms, etc.may be present without falling outside the scope of the correspondingclaim or recitation. As used herein, when the phrase “at least” is usedas the transition term in, for example, a preamble of a claim, it isopen-ended in the same manner as the term “comprising” and “including”are open ended. The term “and/or” when used, for example, in a form suchas A, B, and/or C refers to any combination or subset of A, B, C such as(1) A alone, (2) B alone, (3) C alone, (4) A with B, (5) A with C, (6) Bwith C, and (7) A with B and with C. As used herein in the context ofdescribing structures, components, items, objects and/or things, thephrase “at least one of A and B” is intended to refer to implementationsincluding any of (1) at least one A, (2) at least one B, and (3) atleast one A and at least one B. Similarly, as used herein in the contextof describing structures, components, items, objects and/or things, thephrase “at least one of A or B” is intended to refer to implementationsincluding any of (1) at least one A, (2) at least one B, and (3) atleast one A and at least one B. As used herein in the context ofdescribing the performance or execution of processes, instructions,actions, activities and/or steps, the phrase “at least one of A and B”is intended to refer to implementations including any of (1) at leastone A, (2) at least one B, and (3) at least one A and at least one B.Similarly, as used herein in the context of describing the performanceor execution of processes, instructions, actions, activities and/orsteps, the phrase “at least one of A or B” is intended to refer toimplementations including any of (1) at least one A, (2) at least one B,and (3) at least one A and at least one B.

As used herein, singular references (e.g., “a”, “an”, “first”, “second”,etc.) do not exclude a plurality. The term “a” or “an” entity, as usedherein, refers to one or more of that entity. The terms “a” (or “an”),“one or more”, and “at least one” can be used interchangeably herein.Furthermore, although individually listed, a plurality of means,elements or method actions may be implemented by, e.g., a single unit orprocessor. Additionally, although individual features may be included indifferent examples or claims, these may possibly be combined, and theinclusion in different examples or claims does not imply that acombination of features is not feasible and/or advantageous.

An example program 1000 that may be executed to implement the exampleaudience measurement system 100 of FIG. 1 is illustrated in FIG. 10 .With reference to the preceding figures and associated writtendescriptions, the example program 1000 of FIG. 10 begins execution atblock 1005 at which, if the measurement interval is the firstmeasurement interval for which calibration is to be performed, executionproceeds to block 1010. At block 1010, the example probabilistic panelinterface 105 accesses probabilistic panel data for a calibration marketand for the first measurement interval, as described above. At block1015, the example behavioral distribution estimator 115 determines abehavioral UE distribution for the probabilistic panel data obtained forthe calibration market and for the first measurement interval, asdescribed above. At block 1020, the example calibration distributionselector 120 selects the probabilistic panel behavioral UE distributiondetermined at block 1015 to be the initial calibration behavioraldistribution, as described above. At block 1025, the example conveniencepanel interface 110 accesses convenience panel data for a target marketand for the first measurement interval, as described above. At block1030, the behavioral distribution estimator 115 determines a behavioralUE distribution for the convenience panel data obtained for the targetmarket and for the first measurement interval, as described above. Atblock 1035, the example behavioral distribution calibrator 125 weightsand combines the behavioral UE distribution for the convenience paneldata determined at block 1030 with the initial calibration behavioraldistribution determined at block 1020 to determine a target behavioralUE distribution for the target market and first measurement interval, asdescribed. At block 1040, the example target convenience panel datacalibrator 130 reports the calibrated convenience panel UE distributionfor the target market and first measurement interval, as describedabove.

If processing is to continue for a subsequent measurement interval,execution returns to block 1005. Execution further continues to block1050 because the subsequent measurement interval is not the firstmeasurement interval. At block 1050, the example calibrationdistribution selector 120 selects the target behavioral UE distributiondetermined at block 1040 for the target market and prior measurementinterval to be the initial calibration behavioral distribution for thecurrent measurement interval, as described above. Processing thencontinues through blocks 1025-1040, as described above, which yields atarget behavioral UE distribution for the target market and thesubsequent measurement interval, as described above. Execution thencontinues iterating from block 1045 to block 1050 followed by blocks1025-1040 to yield target behavioral UE distributions for the targetmarket and subsequent measurement intervals.

An example program 1100 that may be executed to implement the exampleaudience measurement system 200 of FIGS. 2-4 is illustrated in FIG. 11 .With reference to the preceding figures and associated writtendescriptions, the example program 1100 of FIG. 11 begins execution atblock 1105 at which the example behavioral distribution initializer 305of the example behavioral distribution estimator 215 included in theaudience measurement system 200 initializes the behavioral distributionparameters used by the behavioral distribute on estimator 215, asdescribed above. At block 1110, the example target market panel dataweighter 310 of the behavioral distribution estimator 215 weights, asdescribed above, convenience panel data from a target market based ongeodemographic weighting parameters for the target market to determineweighted target market convenience panel data (e.g., the target marketconvenience sample A2 described above). At block 1115, the examplecalibration market panel data weighter 315 of the behavioraldistribution estimator 215 weights, as described above, probabilisticpanel data from a calibration market based on demographic weightingparameters for the target market to determine weighted calibrationmarket probabilistic panel data (e.g., the calibration marketprobabilistic sample B1 described above). At block 1120, the calibrationmarket panel data weighter 315 weights, as described above, conveniencepanel data from the calibration market based on the demographicweighting parameters for the target market to determine weightedcalibration market convenience panel data (e.g., the calibration marketconvenience sample B2 described above).

At block 1125, the example behavioral distribution calculator 320 of thebehavioral distribution estimator 215 calculates, as described above, abehavioral UE distribution for the weighted target market conveniencepanel data (e.g., with the category values A2 c% described above). Atblock 1125, the behavioral distribution calculator 320 calculates, asdescribed above, a behavioral UE distribution for the weightedcalibration market probabilistic panel data (e.g., with the categoryvalues B1 c% described above). At block 1125, the behavioraldistribution calculator 320 further calculates, as described above, abehavioral UE distribution for the weighted calibration marketconvenience panel data (e.g., with the category values B2 c% describedabove).

At block 1130, the example preliminary distribution calibrator 405 ofthe example behavioral distribution calibrator 225 included in theaudience measurement system 200 performs, as described above,preliminary calibration of the behavioral UE distribution determined forthe weighted target market convenience panel data (e.g., with thecategory values A2 c%) based on the behavioral UE distributionsdetermined for the weighted calibration market probabilistic panel data(e.g., with the category values B1 c% described above) and the weightedcalibration market convenience panel data (e.g., with the categoryvalues B2 c% described above). At block 1135, the example finaldistribution calibrator 410 of the behavioral distribution calibrator225 performs, as described above, a final calibration of the preliminarycalibrated values (e.g., corresponding to PTBc described above) of theconvenience panel behavioral UE distribution for the target market toyield the final calibrated values (e.g., corresponding to FTBc describedabove) of the convenience panel behavioral UE distribution for thetarget market.

At block 1140, the behavioral distribution estimator 215 and thebehavioral distribution calibrator 225 repeat the processing at blocks1105-1135 to calibrate the convenience panel behavioral UE distributionfor the target market based on probabilistic and convenience panelbehavioral UE distributions for other calibration markets, as describedabove. At block 1140, the behavioral distribution calibrator 225combines (e.g., averages) the calibration results based on the differentcalibration markets to determine a final, calibrated convenience panelbehavioral UE distribution for the target market and the currentmeasurement interval.

At block 1145, the example target convenience panel data calibrator 230of the audience measurement system 200 weights, as described above, theconvenience panel data (e.g., convenience sample) for the target marketand current measurement interval by the geodemographic weightingparameters for the target market and the calibrated behavioral targetsrepresented by the calibrated behavioral UE distribution to determinecalibrated convenience panel data (e.g., a calibrated conveniencesample) for the target market and current measurement interval.Additionally or alternatively, at block 1145, the target conveniencepanel data calibrator 230 weights any other data sample(s) intended torepresent the target market and the current measurement interval by thegeodemographic weighting parameters for the target market and thecalibrated behavioral targets represented by the calibrated behavioralUE distribution to determine corresponding, calibrated data sample(s)for the target market.

At block 1150, the example target convenience panel data calibrator 230of the audience measurement system 200 outputs the final, calibratedconvenience panel behavioral UE distribution for the target market andthe current measurement interval for use in downstream audiencemeasurement processing. Additionally or alternatively, at block 1150,the target convenience panel data calibrator 230 outputs the calibratedconvenience panel data and/or other calibrated data sample(s) for thetarget market for use in downstream audience measurement processing.Additionally or alternatively, at block 1150, the target conveniencepanel data calibrator 230 determines and outputs online media audienceestimate(s) (e.g., ratings, reach, impressions, etc.) based on thecalibrated convenience panel data and/or other calibrated data sample(s)determined for the target market and current measurement interval.

FIG. 12 is a block diagram of an example processor platform 1200structured to execute the instructions of FIGS. 10 and/or 11 toimplement the example audience measurement system 100 and/or the exampleaudience measurement system 200 of FIGS. 1-4 . The processor platform1200 can be, for example, a server, a personal computer, a workstation,a self-learning machine (e.g., a neural network), a mobile device (e.g.,a cell phone, a smart phone, a tablet such as an iPad′), a personaldigital assistant (PDA), an Internet appliance, or any other type ofcomputing device.

The processor platform 1200 of the illustrated example includes aprocessor 1212. The processor 1212 of the illustrated example ishardware. For example, the processor 1212 can be implemented by one ormore integrated circuits, logic circuits, microprocessors, GPUs, DSPs,or controllers from any desired family or manufacturer. The hardwareprocessor 1212 may be a semiconductor based (e.g., silicon based)device. In the illustrated example, when implementing the exampleaudience measurement system 100, the processor 1212 implements theexample probabilistic panel interface 105, the example convenience panelinterface 110, the example behavioral distribution estimator 115, theexample calibration distribution selector 120, the example behavioraldistribution calibrator 125, and the example convenience panel datacalibrator 130. In the illustrated example, when implementing theexample audience measurement system 200, the processor 1212 implementsthe example probabilistic panel interface 105, the example conveniencepanel interface 110, the example behavioral distribution estimator 215,the example behavioral distribution calibrator 225 and the exampletarget convenience panel data calibrator 230.

The processor 1212 of the illustrated example includes a local memory1213 (e.g., a cache). The processor 1212 of the illustrated example isin communication with a main memory including a volatile memory 1214 anda non-volatile memory 1216 via a link 1218. The link 1218 may beimplemented by a bus, one or more point-to-point connections, etc., or acombination thereof. The volatile memory 1214 may be implemented bySynchronous Dynamic Random Access Memory (SDRAM), Dynamic Random AccessMemory (DRAM), RAMBUS® Dynamic Random Access Memory (RDRAM®) and/or anyother type of random access memory device. The non-volatile memory 1216may be implemented by flash memory and/or any other desired type ofmemory device. Access to the main memory 1214, 1216 is controlled by amemory controller.

The processor platform 1200 of the illustrated example also includes aninterface circuit 1220. The interface circuit 1220 may be implemented byany type of interface standard, such as an Ethernet interface, auniversal serial bus (USB), a Bluetooth® interface, a near fieldcommunication (NFC) interface, and/or a PCI express interface.

In the illustrated example, one or more input devices 1222 are connectedto the interface circuit 1220. The input device(s) 1222 permit(s) a userto enter data and/or commands into the processor 1212. The inputdevice(s) can be implemented by, for example, an audio sensor, amicrophone, a camera (still or video), a keyboard, a button, a mouse, atouchscreen, a track-pad, a trackball, a trackbar (such as an isopoint),a voice recognition system and/or any other human-machine interface.Also, many systems, such as the processor platform 1200, can allow theuser to control the computer system and provide data to the computerusing physical gestures, such as, but not limited to, hand or bodymovements, facial expressions, and face recognition.

One or more output devices 1224 are also connected to the interfacecircuit 1220 of the illustrated example. The output devices 1224 can beimplemented, for example, by display devices (e.g., a light emittingdiode (LED), an organic light emitting diode (OLED), a liquid crystaldisplay (LCD), a cathode ray tube display (CRT), an in-place switching(IPS) display, a touchscreen, etc.), a tactile output device, a printerand/or speakers(s). The interface circuit 1220 of the illustratedexample, thus, typically includes a graphics driver card, a graphicsdriver chip and/or a graphics driver processor.

The interface circuit 1220 of the illustrated example also includes acommunication device such as a transmitter, a receiver, a transceiver, amodem, a residential gateway, a wireless access point, and/or a networkinterface to facilitate exchange of data with external machines (e.g.,computing devices of any kind) via a network 1226. The communication canbe via, for example, an Ethernet connection, a digital subscriber line(DSL) connection, a telephone line connection, a coaxial cable system, asatellite system, a line-of-site wireless system, a cellular telephonesystem, etc.

The processor platform 1200 of the illustrated example also includes oneor more mass storage devices 1228 for storing software and/or data.Examples of such mass storage devices 1228 include floppy disk drives,hard drive disks, compact disk drives, Blu-ray disk drives, redundantarray of independent disks (RAID) systems, and digital versatile disk(DVD) drives.

The machine executable instructions 1232 corresponding to theinstructions of FIGS. 10 and/or 11 may be stored in the mass storagedevice 1228, in the volatile memory 1214, in the non-volatile memory1216, in the local memory 1213 and/or on a removable non-transitorycomputer readable storage medium, such as a CD or DVD 1236.

From the foregoing, it will be appreciated that example methods,apparatus and articles of manufacture have been disclosed that performmulti-market calibration of convenience panel data to reduce behavioralbiases. The disclosed methods, apparatus and articles of manufactureimprove the efficiency of using a computing device by enabling largeconvenience panel data samples to be used in audience measurementsystems without introducing behavioral bias to the system, or at leastwith a reduction in the behavioral bias associated with the uncalibratedconvenience panel data samples. The disclosed methods, apparatus andarticles of manufacture are accordingly directed to one or moreimprovement(s) in the functioning of a computer.

Although certain example methods, apparatus and articles of manufacturehave been disclosed herein, the scope of coverage of this patent is notlimited thereto. On the contrary, this patent covers all methods,apparatus and articles of manufacture fairly falling within the scope ofthe claims of this patent.

The following claims are hereby incorporated into this DetailedDescription by this reference, with each claim standing on its own as aseparate embodiment of the present disclosure.

What is claimed is:
 1. An apparatus comprising: a distribution estimatorto: determine a first behavioral distribution for first conveniencepanel data associated with a first market and a first measurementperiod; determine a second behavioral distribution for secondconvenience panel data associated with a second market and the firstmeasurement period, the second market different from the first market;and determine a third behavioral distribution for probabilistic paneldata associated with the second market and the first measurement period;and a distribution calibrator to: calibrate the first behavioraldistribution determined for the first convenience panel data associatedwith the first market based on (i) the second behavioral distributiondetermined for the second convenience panel data associated with thesecond market and (ii) the third behavioral distribution determined forthe probabilistic panel data associated with the second market; andoutput the calibrated first behavioral distribution.
 2. The apparatus ofclaim 1, wherein the distribution estimator includes: a first dataweighter to weight the first convenience panel data associated with thefirst market to determine weighted first convenience panel dataassociated with the first market, the first data weighter to weight thefirst convenience panel data based on at least one of demographicweighting controls or location weighting controls obtained for the firstmarket; and a distribution calculator to determine the first behavioraldistribution based on the weighted first convenience panel dataassociated with the first market.
 3. The apparatus of claim 2, whereinthe distribution estimator further includes: a second data weighter toweight the second convenience panel data associated with the secondmarket to determine weighted second convenience panel data associatedwith the second market, the second data weighter to weight the secondconvenience panel data based on the demographic weighting controlsobtained for the first market; and the distribution calculator is todetermine the second behavioral distribution based on the weightedsecond convenience panel data associated with the second market.
 4. Theapparatus of claim 3, wherein: the second data weighter is to weight theprobabilistic panel data associated with the second market to determineweighted probabilistic panel data associated with the second market, thesecond data weighter to weight the probabilistic panel data based on thedemographic weighting controls obtained for the first market; and thedistribution calculator is to determine the third behavioraldistribution based on the weighted probabilistic panel data associatedwith the second market.
 5. The apparatus of claim 1, wherein thedistribution calculator includes a first distribution calculator and asecond distribution calculator, the first distribution calculator tocalibrate a value of a first category of the first behavioraldistribution based on a corresponding value of the first category of thefirst behavioral distribution and a corresponding value of the firstcategory of the third behavioral distribution to determine a firstcalibrated value of the first category of the first behavioraldistribution.
 6. The apparatus of claim 5, wherein the firstdistribution calculator is to calibrate the value of the first categoryof the first behavioral distribution by scaling the value of the firstcategory of the first behavioral distribution by a ratio of (i) thecorresponding value of the first category of the first behavioraldistribution to (ii) the corresponding value of the first category ofthe third behavioral distribution.
 7. The apparatus of claim 5, whereinthe second distribution calculator is to calibrate the first calibratedvalue of the first category of the first behavioral distribution todetermine a second calibrated value of the first category of the firstbehavioral distribution.
 8. The apparatus of claim 1, further includinga data calibrator to calibrate the first convenience panel dataassociated with the first market based on the calibrated firstbehavioral distribution.
 9. A non-transitory computer readable mediumcomprising computer readable instructions that, when executed, cause aprocessor to at least: determine a first behavioral distribution forfirst convenience panel data associated with a first market and a firstmeasurement period; and determine a second behavioral distribution forsecond convenience panel data associated with a second market and thefirst measurement period, the second market different from the firstmarket; determine a third behavioral distribution for probabilisticpanel data associated with the second market and the first measurementperiod; calibrate the first behavioral distribution determined for thefirst convenience panel data associated with the first market based on(i) the second behavioral distribution determined for the secondconvenience panel data associated with the second market and (ii) thethird behavioral distribution determined for the probabilistic paneldata associated with the second market; and output the calibrated firstbehavioral distribution.
 10. The non-transitory computer readable mediumof claim 9, wherein the instructions, when executed, cause the processorto: weight the first convenience panel data associated with the firstmarket to determine weighted first convenience panel data associatedwith the first market, the first convenience panel data to be weightedbased on at least one of demographic weighting controls or locationweighting controls obtained for the first market; and determine thefirst behavioral distribution based on the weighted first conveniencepanel data associated with the first market.
 11. The non-transitorycomputer readable medium of claim 10, wherein the instructions, whenexecuted, cause the processor to: weight the second convenience paneldata associated with the second market to determine weighted secondconvenience panel data associated with the second market, the secondconvenience panel data to be weighted based on the demographic weightingcontrols obtained for the first market; and determine the secondbehavioral distribution based on the weighted second convenience paneldata associated with the second market.
 12. The non-transitory computerreadable medium of claim 11, wherein the instructions, when executed,cause the processor to: weight the probabilistic panel data associatedwith the second market to determine weighted probabilistic panel dataassociated with the second market, the probabilistic panel data to beweighted based on the demographic weighting controls obtained for thefirst market; and determine the third behavioral distribution based onthe weighted probabilistic panel data associated with the second market.13. The non-transitory computer readable medium of claim 9, wherein theinstructions, when executed, cause the processor to: calibrate a valueof a first category of the first behavioral distribution based on acorresponding value of the first category of the first behavioraldistribution and a corresponding value of the first category of thethird behavioral distribution to determine a first calibrated value ofthe first category of the first behavioral distribution; and furthercalibrate the first calibrated value of the first category of the firstbehavioral distribution to determine a second calibrated value of thefirst category of the first behavioral distribution.
 14. Thenon-transitory computer readable medium of claim 13, wherein theinstructions, when executed, cause the processor to calibrate the valueof the first category of the first behavioral distribution by scalingthe value of the first category of the first behavioral distribution bya ratio of (i) the corresponding value of the first category of thefirst behavioral distribution to (ii) the corresponding value of thefirst category of the third behavioral distribution.
 15. A methodcomprising: determining, by executing an instruction with a processor, afirst behavioral distribution for first convenience panel dataassociated with a first market and a first measurement period; anddetermining, by executing an instruction with the processor, a secondbehavioral distribution for second convenience panel data associatedwith a second market and the first measurement period, the second marketdifferent from the first market; determining, by executing aninstruction with the processor, a third behavioral distribution forprobabilistic panel data associated with the second market and the firstmeasurement period; calibrating, by executing an instruction with theprocessor, the first behavioral distribution determined for the firstconvenience panel data associated with the first market based on (i) thesecond behavioral distribution determined for the second conveniencepanel data associated with the second market and (ii) the thirdbehavioral distribution determined for the probabilistic panel dataassociated with the second market; and outputting the calibrated firstbehavioral distribution.
 16. The method of claim 15, wherein thedetermining of the first behavioral distribution includes: weighting thefirst convenience panel data associated with the first market todetermine weighted first convenience panel data associated with thefirst market, the first convenience panel data being weighted based onat least one of demographic weighting controls or location weightingcontrols obtained for the first market; and determining the firstbehavioral distribution based on the weighted first convenience paneldata associated with the first market.
 17. The method of claim 16,wherein the determining of the second behavioral distribution includes:weighting the second convenience panel data associated with the secondmarket to determine weighted second convenience panel data associatedwith the second market, the second convenience panel data being weightedbased on the demographic weighting controls obtained for the firstmarket; and determining the second behavioral distribution based on theweighted second convenience panel data associated with the secondmarket.
 18. The method of claim 17, wherein the determining of the thirdbehavioral distribution includes: weighting the probabilistic panel dataassociated with the second market to determine weighted probabilisticpanel data associated with the second market, the probabilistic paneldata being weighted based on the demographic weighting controls obtainedfor the first market; and determining the third behavioral distributionbased on the weighted probabilistic panel data associated with thesecond market.
 19. The method of claim 15, wherein the calibrating ofthe first behavioral distribution includes: calibrating a value of afirst category of the first behavioral distribution based on acorresponding value of the first category of the first behavioraldistribution and a corresponding value of the first category of thethird behavioral distribution to determine a first calibrated value ofthe first category of the first behavioral distribution; and furthercalibrating the first calibrated value of the first category of thefirst behavioral distribution to determine a second calibrated value ofthe first category of the first behavioral distribution.
 20. The methodof claim 19, wherein the calibrating of the value of the first categoryof the first behavioral distribution includes scaling the value of thefirst category of the first behavioral distribution by a ratio of (i)the corresponding value of the first category of the first behavioraldistribution to (ii) the corresponding value of the first category ofthe third behavioral distribution.